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Abstract 


Using data obtained from interviews and program websites at Washington 
community and technical colleges, the authors of this study examine the structure of 
community college career-technical programs in allied health, business and marketing, 
computer and information studies, and mechanics and repair. A framework for structure 
with four dimensions — program alignment, program prescription, infonnation quality, 
and active program advising and support — is used to evaluate the practices of relatively 
high- and low-perfonning colleges within each field of study. The authors reviewed the 
websites of all programs at high- and low-perfonning colleges in each of these fields of 
study and conducted case studies on individual programs from these fields, interviewing 
faculty, administrators, and counselors to leam more about the dimensions of structure in 
the programs. The allied health, computer and infonnation science, and mechanics and 
repair programs were all found to be highly structured; the business and marketing 
programs were found to have a moderate level of structure. Overall, given that all of the 
programs were at least moderately structured, there was limited evidence of a connection 
between program structure and program performance. 
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1. Introduction 


Increasing the number of postsecondary credentials, particularly those earned in 
community colleges, is a major national goal. However, current graduation rates from 
community colleges are low: Fewer than 36 percent of first-time community college 
students in 2003-04 completed a program of study and attained a credential within six 
years (Radford, Berkner, Wheeless, & Shepherd, 2010). To meet national goals for 
credential attainment, significant refonn is needed within community colleges to increase 
graduation rates. 

One possible area of reform is the way programs of study are structured to 
promote student completion. Based on a review of research on college student success 
and the behavioral economics literature, Scott-Clayton (2011) hypothesized that 
“community college students will be more likely to persist and succeed in programs that 
are tightly and consciously structured, with relatively little room for individuals to 
unintentionally deviate from paths towards completion, and with limited bureaucratic 
obstacles for students to circumnavigate.” Rosenbaum, Deil-Amen, and Person (2006) 
developed a concept of structure similar to Scott-Clayton’s in their study of a sample of 
community colleges and occupational colleges (both for-profit and not-for-profit). They 
identified students’ lack of knowledge about college procedures and realistic pathways to 
graduation as a barrier to completion for students in the community colleges. In contrast, at 
the occupational colleges, they found very structured programs that allowed little choice 
and provided a clear sequence of courses. Limiting choice creates less room for mistakes 
that may prevent students from reaching their goals. Rosenbaum et al. also found that the 
advising offered at the occupational colleges was more intensive and mandatory for all 
students, compared with the optional and often inaccessible advising offered at the 
community colleges. Moreover, at the occupational colleges, peer cohorts progressed 
through programs together, providing students with additional structure and support. 

In addition, Rosenbaum et al. (2006) found that the community colleges lacked 
strong labor market linkages and did not prepare students well for finding employment. 
Unlike occupational colleges, the community colleges had weak advisory boards (if any) 
that met infrequently and were not composed of employers that were likely to hire 
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graduates. Job placement efforts at the community colleges were haphazard rather than 
active with dedicated staff, and community college staff did not emphasize social skills 
necessary for employment, such as appropriate workplace attire and punctuality. 

This research from Rosenbaum et al. (2006) suggests that the structure of college 
programs influences student success. Its value for understanding program structure in 
community colleges, however, is limited. Many of Rosenbaum et al.’s criticisms of 
community colleges are based on analysis of the full range of programs in community 
colleges, including liberal arts and sciences, rather than on analysis of the specific career- 
technical 1 programs that are most comparable to those offered by occupational colleges. 2 3 
Prior research suggests that students in liberal arts and career-technical programs may 
have very different characteristics and intents (Jenkins & Weiss, 2011). In this paper, we 
examine how program structure may affect student success in the specific context of 
community college career-technical programs. 

We build on Scott-Clayton’s (2011) and Rosenbaum et al.’s (2006) conceptions of 
structure in order to examine the structure of community college career-technical 
programs in Washington State. Although we recognize that the institutional practices 
associated with higher levels of structure may exist at both the college level and the 
program level, in the current study, we focus primarily on program-level practices, and 
specifically on how programs are organized and how information about the programs and 
their requirements is conveyed to students. Program-level practices can help students 
complete a credential quickly and efficiently and can also help facilitate the recruitment 
process by drawing prospective students directly into the program and onto the path 
toward a credential. 


1 We refer to programs designed to prepare students for direct entry into the labor market (as opposed to 
general liberal arts programs designed for baccalaureate transfer) as “career-technical programs.” 
Elsewhere, these programs are sometimes referred to as “workforce” or “occupational” programs. 

2 The research design used by Rosenbaum et al. did not allow for the systematic comparison of similar 
programs across the two sectors, though they made some attempts in that direction. Although the student 
survey they conducted was administered to students in similar introductory classes across the sectors, the 
student interview sample (which appeared to be heavily relied upon for some conclusions) was not. 
Moreover, as the authors note, the selection of colleges was not random within each sector but rather 
included some well-regarded private sector colleges and some relatively low-performing community 
colleges, such that generalizations about the sectors from their sample may not be warranted. 

3 College-level practices relating to structure include recruitment, orientation, and counseling and advising 
practices that relate to students’ choice of program. 
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To understand these practices from the perspective of the college as an institution, 
we developed a framework of program structure, drawing on previous literature. This 
framework of program structure has four dimensions: program alignment, program 
prescription, information quality, and active program advising and support. Program 
alignment and program prescription both relate to how programs and fields of study are 
organized — that is, the choices made at the colleges about what comprises a program and 
field of study. Program alignment refers to how the program is linked to employment and 
further educational outcomes. Program prescription refers to whether the program 
requirements are clearly specified and the level of flexibility students have in choosing 
their courses. Information quality refers to the quality of the informational resources 
available to students. Active advising and student support refers to the actions college 
staff take to convey program information to students and keep students engaged in — and 
progressing through — their programs. 

With this framework, we aim to understand how community and technical college 
career-technical programs in Washington State are organized and how that organization 
may affect students. In this paper, we address the following research questions: 

1 . How closely are programs aligned with further educational and 
employment opportunities, and to what extent does this 
alignment vary across programs in practice? 

2. How prescribed are program requirements, and how much 
variation in program prescription is there across programs 
in practice? 

3. What is the quality of infonnation available to current and 
prospective students, both through the college website and other 
means, and how does this vary across programs in practice? 

4. What types of advising and student supports are offered to 
current and prospective students? 

5. Is there any relationship between the degree of structure along 
these dimensions and program perfonnance, defined by 
completion rates of long-term certificates and associate degrees? 

In the next section of this paper, we describe our methodology in our field-of- 
study and site selection and subsequent fieldwork. In the following sections, we examine 
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the manner in which our four dimensions of structure are reflected and implemented in 
our sample of programs. We then consider how structure might be associated with 
program performance. Finally, we discuss the findings of this study and our 
recommendations for colleges and for future research. 


2. Methods 

We pursued our research questions in three stages. Using administrative data from 
the state of Washington, we selected four fields of study for further qualitative research 
and selected relatively high- and low-performing colleges within each field of study. We 
then reviewed the websites of all programs at high- and low-perfonning colleges in each 
of these fields of study, noting the program requirements and the quality of information 
available to students. Finally, we conducted case studies on high-performing and low- 
performing programs in each field, interviewing faculty, administrators, and counselors 
to learn more about the dimensions of structure in the programs. 

2,1 Administrative Data and Site Selection 

To select our sample of programs, we used data from administrative records 
provided through a data sharing agreement by the Washington State Board for 
Community and Technical Colleges (SBCTC). These data included transcript records and 
demographic infonnation for all first-time students at Washington community and 
technical colleges who began their studies during the 2005-06 academic year. 

We used transcript records to select fields of study and colleges for our qualitative 
research. We distinguish here between fields of study and programs. Fields of study are 
broad sets of programs that generally fall within two-digit Classification of Instructional 
Programs (CIP) code categories, such as business and marketing or allied health. 4 

4 Using a taxonomy adapted from the National Center for Education Statistics, we categorized courses in 
students’ transcripts into one of 22 fields based on their CIP codes: arts, humanities, and English; mathematics 
and science (STEM); social and behavioral sciences; agriculture and natural resources; automotive and 
aeronautical technology; business and marketing; secretarial and administrative studies; communications 
and design; computer and information science; cosmetology; culinary services; engineering and architecture; 
engineering/science technologies; education and child care; allied health; nursing; construction; 
manufacturing; mechanics and repair; transportation; protective services; and other career-technical. 
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Programs are groupings of courses that lead to a specific credential; they are subsets of 
broader fields of study. For example, an accounting long-term certificate is a program 
within the business and marketing field of study. 5 

We identified fields of study with relatively high enrollments across the 
Washington community and technical colleges. In order to have enough data to perfonn 
analyses for the selection of sites for our later qualitative research, we focused on 
colleges where at least 20 students were served at a time within a given field of study. For 
each field of study, we identified how many out of the 34 Washington community and 
technical colleges served at least 20 students attempting a concentration in that field; 
Table 1 shows a list of these fields of study, ranked by the number of colleges with 20 or 
more students enrolled. For our analyses, we selected the four fields with the most such 
colleges: business and marketing; computer and information science; allied health; and 
mechanics and repair. Because we focus on issues that take place at the program level, 
we limited our sample to students who attempted a concentration in a particular field of 
study, taking at least three classes or twelve quarter credits within a single field. 6 

After selecting fields of study, we sought to identify relatively high- and low- 
performing colleges in each field so that we might observe a diverse set of policies 
concerning program structure. We did not make direct comparisons between high- and 
low-performing programs due to concerns that this analysis (conducted at the field-of- 
study level) might not effectively measure performance of the specific programs, which 
we later examine in case studies. However, given the absence of a better program-level 
measure, we used these perfonnance rankings in our site selection process and in making 
some tentative observations about the relationship between program structure and student 
outcomes. Appendix A details our methods in ranking colleges along program 
performance, and Section 4 discusses the specific problems with the selection 
methodology. Ultimately, for each field of study, we ranked all colleges with at least 20 

5 At some colleges, several credentials may be packaged together and taught by the same group of faculty 
members, such that a student might earn several certificates en route to an associate degree. Though some 
colleges might refer to this entire package of credentials as a “program,” in this paper, we refer to this as a 
“program pathway,” as distinguished from a program that leads to a specific credential. 

6 Washington State colleges operate on the quarter system, and students must take at least twelve credits in 
a given quarter to be considered full-time students. Thus, under our definition, a concentrator is a student 
who attempts at least one term’s worth of coursework in a given field of study. 
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concentrators from the 2005-06 cohort in that field based on how well they perfonned 
relative to expected completion rates of long-term certificates and associate degrees and 
used this list to select colleges for the website review and interview stages of our study. 


Table 1 

Fields of Study, Ranked by Number of Colleges With 20 or More Students Enrolled 


Field of Study 

Rank 

Number of Colleges with 
20+ Students Enrolled 

Total Number of 
Students Enrolled 

Business and marketing 

1 

23 

2,259 

Computer and information sciences 

2 

22 

1,041 

Allied health 

3 

21 

1,140 

Mechanics and repair 

4 

15 

835 

Manufacturing 

5 

11 

645 

Education and child care 

6 

10 

724 

Construction 

7 

10 

1,286 

Protective services 

8 

8 

364 

Cosmetology 

9 

5 

273 

Engineering/science technologies 

10 

5 

536 

Secretarial and administrative services 

11 

4 

278 

Culinary services 

12 

4 

271 

Transportation 

13 

4 

207 

Agriculture and natural resources 

14 

3 

250 

Communications and design 

15 

3 

213 

Nursing 

16 

3 

366 

Automotive and aeronautical technology 

17 

0 

17 

Engineering and architecture 

18 

0 

30 

Other career-technical 


6 

421 

Not assigned 


16 

789 

Arts, humanities, and English 


29 

6,746 

Mathematics and science (STEM) 


28 

2,284 

Social and behavioral sciences 


23 

1,775 


Source: SBCTC administrative data. 

Note. Students whose primary intent was vocational home and family life (that is, to take parenting classes) were 
excluded from the analysis, as were students obtaining food cards (licenses to handle food in Washington that require 
a short course). 
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2.2 Website Review 

In the summer of 201 1, we reviewed the websites of programs in the four fields of 
study across 20 colleges that were relatively high-performing or low-performing within 
the field of study. We collected data on all programs at the college within each field of 
study, as defined by the CIP codes. For example, the computer and information sciences 
field at a given college might include several distinct program pathways: computer 
applications technology, infonnation technology, and multimedia communications. 

Through the colleges’ websites, we collected information on program prescription 
(that is, the amount of choice and specificity in requirements in the programs). Because 
the websites are an important way that colleges convey information on programs to 
students, we assessed the quality of information on colleges’ programs on their websites, 
looking for availability and ease of accessibility. 

2.3 Interviews 

Following the website review, we chose one high-perfonning program pathway 
and one low-performing program pathway within each of the four fields of study, 
ensuring that the two colleges selected offered comparable program pathways. Choosing 
comparable programs allowed us to focus on differences in their structure rather than 
differences in course content. A program pathway in diesel and heavy equipment, for 
example, might be too substantively different from a program pathway in marine 
maintenance technology to allow us to draw conclusions about program structure from 
differences observed between them, even though both pathways fall into the broad 
category of mechanics and repair. Table 2 displays the program pathways we highlighted 
from each field of study for these case studies. 

During the fall of 20 1 1 , we interviewed key college staff involved with each of 
the eight programs, including an academic dean, preferably in charge of career-technical 
programs; a department chair in the selected field of study; at least two faculty members 
in the program; and a counselor or advisor from student services/counseling. The semi- 
structured telephone interviews were typically one hour in length and covered several 
topics, including program alignment, program prescription, information quality, and 
active advising and student support. 
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Table 2 

Program Pathways Selected for Case Study Within Each Field of Study 


Field of Study 

Program Pathway 

Allied health 

Medical assisting 

Business and marketing 

Accounting 

Computer and information sciences 

Computer network technology 

Mechanics and repair 

Automotive technology 


3. Findings 

In this section, we highlight findings from the three stages of data collection and 
analysis and address our five research questions. We first describe findings related to 
each of the four dimensions in our framework of program structure (program alignment, 
program prescription, information quality, and student advising and support) and then 
examine the relationship between program structure and performance. 

3,1 Program Alignment 

Program alignment refers to the ways in which programs of study are 
intentionally structured to be well aligned with pathways and institutions outside the 
college. Program alignment is an important part of program structure because it 
influences both the curriculum that students are exposed to while in the program and 
students’ academic and career opportunities upon graduation. In examining this 
dimension of structure, we rely on data from our case studies of individual programs. 

Case studies of individual programs. Through our research, we identified 
three primary manifestations of program alignment. Labor market alignment refers to 
connections with industry and employers that affect the content and format of a 
program. Alignment with local employment opportunities refers to ways in which a 
program directly prepares students to enter the local entry-level job market. 

Educational alignment refers both to structure of programs within the program pathway 
at the college and to the creation of transfer pathways that connect students to other 
academic institutions. 


8 



These three types of alignment may often intersect, and within different 
programs, different values may be placed on each type of alignment. In addition, there 
are a variety of ways to achieve each type of alignment (see Table 3 for details), and 
tradeoffs are often made between them. For example, in a program with a strong, 
mandatory clinical component that often leads to a job after graduation, students may be 
less likely to be offered other job placement assistance. As another example, in a program 
where labor market alignment is not maintained via a national accreditation body, labor 
market alignment may be achieved through close connections with an advisory board of 
local employers. 

Labor market alignment. Strong influence of accreditation bodies on college 
programs may increase prescription and limit choices about what content to offer and 
how to structure the program. At least one interviewee argued that accreditation is such a 
powerful influence on some programs that it splits occupational programs into two 
camps: those that are accredited by an outside body and those that are not. This 
administrator stated: 

Programs [at my college] that have individual accreditations (like 
Occupational Therapy Assistant, Physical Therapist Assistant, 

Funeral Services, Dental Assisting) — those accrediting bodies 
really put them through the sieve as it were. They really drive the 
limitations of what a program can do, because [the accreditation 
bodies] are becoming more and more prescriptive in their 
requirements. Whereas computer network technology is much 
more closely aligned with their industry through their advisory 
committees, there may be industry certifications that apply, but 
there isn’t a regional or national accrediting body for them. That’s 
a really big one that creates two different camps of programs. 

Labor market alignment varies more across fields of study than across colleges 
within fields of study. However, there is some variation within fields in the strength of 
the advisory board. In general, programs in the field with the most rapidly changing 
technologies seem to rely most heavily on their advisory boards. Faculty in both 
computer network technology programs used their advisory boards to keep up with 
changes in industry standards, modifying the curriculum to incorporate new developments 
such as cloud computing and Voice over Internet Protocol on the advice of the boards. 
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Table 3 

Manifestations of Program Alignment in Individual Programs 
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Advisory boards also played a significant role in the automotive programs, where they 
were mandated by the accreditation process but considered essential to the mission of 
ensuring students are well prepared for jobs. The advisory boards of the medical assisting 
programs, in contrast, had only moderate influence, perhaps due to tight national 
accreditation standards, which limited the extent to which advisory boards could 
influence the program further. Accounting seems to have much less labor market 
alignment under our definition, though this may be because the field has not changed 
drastically over time. However, one accounting program did add 10-key typing to the 
program following the advice of its advisory board. 

Alignment with local employment opportunities. All of the occupational 
programs we examined were designed to prepare students for employment. However, the 
programs differed in the approaches used to help students make a seamless transition into 
the working world. The majority of programs incorporated some kind of hands-on 
training to prepare students for the type of work they would be doing upon graduation. 
This component was the strongest in medical assisting, where a clinical experience was 
mandatory and often led to job placement after graduation. Only one other program — one 
of the automotive programs — required an internship as part of the curriculum. This 
requirement could be fulfilled in one of two ways: A student’s regular job or as little as 
50 hours of job shadowing could both qualify. Several other programs offered internships 
for credit, but program staff reported they were rare. A computer networking technology 
faculty member at one such program said that program staff did not focus on matching 
students with internships, noting that it seemed like many companies offering internships 
“just wanted someone to do work and not pay them.” It is also worth noting that two 
programs (one automotive program and one accounting program) did not offer 
internships due to lack of placement opportunities. However, in both programs, a strong 
lab component (with a similar environment to what students would experience upon 
beginning entry-level employment) was incorporated into the daily schedule. 

In addition to providing hands-on training, most programs helped to prepare 
students in some way for the job search. One automotive program required a three-credit 
course in job seeking skills as part of its program, and one computer network technology 
program offered a two-credit elective course on resumes and interviews. The other 
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automotive and computer network technology programs, as well as the accounting 
programs, did not offer an explicit course but incorporated some career guidance into the 
curriculum by preparing the students with mock interviews, including information on 
resume preparation, or both. 

Faculty members differed in their degree of involvement in job placement for 
students. At one medical assisting program, the program director did not have time for 
placements; the program director at the other did some informal networking with 
program graduates to assist them with their job search. Neither accounting program had 
faculty members actively assisting with job placement, though both colleges had campus- 
wide job placement offices. Automotive faculty members, in contrast, were very active in 
job placement; interviewees at both programs reported that faculty members frequently 
helped students find a good job fit. Computer network technology faculty members were 
moderately active, sending out information about available job opportunities in class or 
via online social networks as they came up. 

All colleges received data from the state on the employment outcomes of their 
graduates, although there was variation in the extent to which this information was 
valued, considered timely and accurate, and used. Colleges also varied in the extent to 
which they performed their own tracking of students’ employment outcomes. Whether 
employment tracking was a priority for colleges was heavily influenced by external 
requirements. The accrediting bodies for the medical assisting and automotive technology 
programs required careful tracking of employment placement and success. For example, 
one allied health program performed 30-day and 90-day reviews of graduates’ 
employment placements and outcomes. Other colleges used more informal methods of 
tracking students, such as optional online social networks for alumni in the computer 
network technology programs. 

Educational alignment. “Stackable” credentials — that is, credentials that can be 
earned along the way to a longer term credential — were an important component of the 
occupational programs we studied. All eight programs offered at least one certificate that 
could be earned along the way to an associate degree. In particular, each program offered 
either a certificate earned after the first year of program requirements or a certificate 
earned after the completion of all program courses (except general education 
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requirements). However, there was a great deal of variety in how many other 
opportunities there were to earn shorter tenn certificates in specific areas along the way 
to the associate degree. Three programs — across three different fields of study — offered 
no further opportunities for stacking credentials. Others offered a range of additional 
options, from a single short three-course accounting clerk certificate at one school to 
seven specialized short-tenn automotive technology certificates at a different school that 
could be earned either along the way to an associate degree (sometimes with some 
limited additional coursework) or separately. The diversity in types of certificates offered 
was largely explained by whether certificates were designed to be aligned with external 
industry standards (as in the case of the specialized short-term automotive technology 
certificates) or whether they were designed as a potential stopping point for students 
without a clear labor market linkage. 

Programs and colleges differed in their emphasis on alignment with further 
education, both in terms of encouraging associate degrees and offering opportunities for 
baccalaureate transfer. This could be thought of as a tension between educational 
alignment and alignment with local employment opportunities. Programs that emphasized 
the fonner could be considered to have an educational orientation, whereas programs that 
emphasized the latter could be considered to have an employment orientation. Programs 
with an educational orientation are likely to emphasize the importance of earning an 
associate degree and possibly a bachelor’s degree. For example, the college that ran both 
Program 2 and Program 6 (in medical assisting and computer network technology) 
encouraged the students in each program to earn the associate degree. Faculty in the 
college’s computer network technology program encouraged students to aspire to a 
bachelor’s degree and worked to create strong articulation agreements with nearby 
colleges and universities, anticipating that students may face barriers in career mobility 
later in life without the opportunity to earn these additional credentials. In contrast, 
programs with an employment orientation were likely to emphasize skills needed for 
immediate job placement, even if doing so may lead to fewer opportunities for additional 
education. For example, not as many students in Program 1 (medical assisting) earned an 
associate degree because the certificate leads to strong employment opportunities, and 
certificate holders and associate degree holders earn similar wages. In Program 5 


13 



(computer network technology), the associate degree was considered a good option for 
students who could not otherwise find jobs. Little to no thought was given to improving 
students’ prospects for baccalaureate transfer because a bachelor’s degree is not 
considered necessary in the labor market. 

Given that all of the programs we examined were occupational programs, 
baccalaureate transfer was rare even among schools that had an articulation agreement. 
Both automotive programs had developed an articulation agreement with a four-year 
college in a neighboring state for a bachelor of science degree in automotive technology, 
but neither reported the degree as being common for students. Similarly, Program 3 
(accounting) had an articulation agreement with another college in the state, but students 
rarely transferred. Transfer was slightly more common in Program 6 (computer network 
technology), which was in the process of finalizing an articulation agreement with a state 
university for a bachelor’s degree in technology and design. Meanwhile, another nearby 
college was developing a four-year program in information technology for the health 
industry where Program 6 graduates could enroll as juniors. However, the use of 
articulation agreements may be limited by the ability of students — particularly students 
with low socioeconomic status — to relocate for educational opportunities that are not 
within commuting distance. 

Overall assessment. In general, the career-technical programs we examined were 
tightly aligned with labor markets and local employers through accreditation agencies 
and advisory boards. This was somewhat less true in the accounting programs than in 
other program areas. Additionally, all of the programs provided students with some 
assistance in job search skills and job placement, often integrating job search skills 
directly into the curriculum. In some cases, faculty members used their connections to 
help students find local employment opportunities. However, programs held diverse 
attitudes toward alignment with further educational opportunities. At some colleges, the 
career-technical program was seen as a terminal degree and further education seen as 
unnecessary, and at others, it was seen as important to give students the option to pursue 
further education. Views on this issue differed even between programs with similar 
content at different colleges. 
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3.2 Program Prescription 

Program prescription refers to how clearly a program’s requirements are 
delineated and how flexible those requirements are. Highly prescribed programs aim to 
prevent students from enrolling in unnecessary coursework. A highly prescribed program 
might include more required program courses than electives; use a cohort model, in 
which students proceed through the coursework as a group; integrate general education 
courses into the technical program; and schedule courses with intentionality both within 
and across semesters. 

Website review. A key goal of our website review was to better understand 
program requirements within our four fields of study. College websites should list all 
required courses for each program, and we collected this information for all programs 
within each college within a given field of study. In particular, we recorded the total 
number of credits required to attain a credential in each program and the distribution of 
these credits across four categories: required program classes, required general 
education classes, program electives, and general education or other electives. The 
examination of program requirements provides an indication of program prescription as 
reflected in the degree of choice students are permitted in selecting their courses. A 
more prescribed program would have a greater proportion of required credits that are 
specifically mandated, as opposed to allowing room for electives. Additionally, a more 
prescribed program would have more of its required credits fall within the program area 
itself, as opposed to within the general education curriculum. The concept of prescription 
could include additional issues such as whether courses are required to be taken in a 
particular sequence. 7 In the following summary of results, program requirements are 
reported separately for long-tenn certificates and associate degrees, which have different 
credit requirements. 

Across all fields of study, the majority of credits necessary to earn a credential 
were in required program courses (see Figures 1 and 2, which reflect program 
requirements as reported on the 20 college websites we reviewed). In long-term 
certificate programs the vast majority of courses were specifically mandated courses 

7 Though some college websites reported information about recommended sequencing, this information 
was not reported consistently enough across program websites to compare programs along this measure. 
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within the program area. Associate degree programs had more general education 
requirements and elective courses but were still dominated by required program 
courses. Overall, community and technical college career-technical programs in these 
four fields of study did not tend to include a high number of electives or general 
education requirements. 


Figure 1 

Program Requirements in Long-Term Certificate Programs by Field 
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Figure 2 

Program Requirements in Associate Degree Programs by Field 
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In addition to counting the number of required and elective credits, we assessed 
the degree of program prescription using a rubric (see Table 4 for overall ratings and 
Table 5 for details). This assessment was based on a review of all required and elective 
courses, regardless of whether they came from the program area or from the general 
education curriculum. Programs were coded as very prescribed, fairly prescribed, or 
somewhat prescribed, depending on how much choice students had in their course 
selection. For example, a program that allowed students to choose one program elective 
from three options and to choose between two social science courses to meet the general 
education requirement might be coded as fairly prescribed. Theoretically, we could have 
coded programs as not very prescribed, but none of the programs we examined fell into 
that category. 
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Table 4 

Assessment of Program Prescription in Career-Technical Programs by Field and Type of Credential 
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Across fields of study, most programs were very prescribed. This assessment is 
consistent with our finding that a large number of credits in each program were required 
program credits (see Figures 1 and 2). In general, long-term certificate programs were 
more prescribed than associate degree programs within the same field. Nearly all long- 
term certificate programs were rated very prescribed; there was slightly more variation 
among associate degree programs. 

Case studies of individual programs. Findings from the case studies of two 
programs within each field of study were generally consistent with the results from the 
website review. Through interviews conducted with faculty, counselors, and 
administrators, we found all of the programs to be very clearly prescribed, with few or no 
elective courses. For most programs, the majority of the courses required for the associate 
degree were in the technical program area as opposed to general education courses. 
Among all the courses required for an associate degree, both program and general 
education, very few were left up to the choice of the students. 

Cohort models . Cohort models, in which students both enter a program and take 
all subsequent courses together, were used in five of the eight programs we examined 
(with the exceptions of the two accounting programs and one allied health program). In 
the automotive programs and the allied health program with a cohort, the cohort model 
was largely a result of the very prescribed nature of these programs as determined by 
external accreditation standards. In computer network technology, faculty preferred the 
cohort model because it allowed them to incorporate hands-on lab experiences and to 
devise course content that progressively builds upon previous course material. 

College staff reported that the cohort model creates bonds between students, who 
support each other as they progress through the program. Furthermore, it allows faculty 
to get to know the students better and to identify problems more easily. However, several 
college staff mentioned that the cohort model can have downsides, including a lack of 
flexibility in students’ course schedules to accommodate work or family obligations. The 
cohort model programs required a full-time commitment, precluding students who had 
substantial work commitments from enrolling. College staff reported that this full-time 
commitment helped students to progress through the program and be more focused than 
those who also had job commitments. 
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Sequencing/degree of rigidity . Some cohort models limited the points in the year 
when students could enter the program so that cohorts of students could enter together. 
Although more frequent entry points allow students to spend less time waiting to begin 
enrollment, not all programs have enough faculty to run multiple cohorts per year. Some 
programs allowed for flexibility in when students could enter the programs, even if there 
were a rigid entry point in theory. The computer network technology and automotive 
programs made some exceptions to allow students to enter into an existing cohort. In 
these cases, although enrollment was flexible, it was not completely open. For example, 
in one of the automotive programs, students could enroll in the winter term after the 
cohort began in the fall, but they could not join that cohort in the spring term. Programs 
that did not use the cohort model allowed students to begin the program at any time 
during the year. The accounting programs in particular were very flexible in this respect 
and also allowed students to take courses in the order that would best suit their schedules. 

All programs that used a cohort model had a very rigid sequence of required 
courses. Often, if a student missed taking a course with the cohort or did not pass a 
course in the required sequence, the student could not retake the course until it was 
offered the next year. In some cases, students could make up courses over the summer 
tenn. Many program staff working with cohorts reported that faculty got to know 
students well and could intervene to help struggling students, so that few students would 
not pass required courses. However, rigid course sequences might limit the types of 
students who are able to enroll in the program because students who need the flexibility 
to stop out of college for non-academic reasons and resume their education later would be 
unable to do so. 

Rigid course sequencing is often driven by the cohort model but may also exist in 
programs that do not use cohorts. One allied health program without a cohort model had 
fairly rigid sequencing requirements because many of the required courses built on each 
other. Because of the abundance of prerequisites among the program courses, students 
had to take courses in a specific sequence, and program staff provided guidance on how 
to do this through group advising sessions each term before course registration. In 
contrast, the accounting programs, which also had no cohorts, offered students a great 
deal of flexibility in the sequence in which they could take the courses. One of these 
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programs offered a suggested course sequence but did not mandate that students follow 
it; the other program provided students with a checklist of courses to take but no 
recommended sequence. 

Integration of general education requirements. Whether programs integrated 
general education courses into the sequence of program courses varied across colleges. In 
several programs, general education courses were only part of the required sequence of 
courses when they were prerequisites for program entry. In one medical assisting 
program, many students took general education requirements while on the waiting list for 
the program; others took those courses after completing the program courses. A couple of 
the programs recently moved to make the general education courses part of the required 
sequence of courses so that students would not delay taking them until the end of their 
program, when they would be less likely to complete them. 

Intentionality in course scheduling. The extent to which faculty in the programs 
coordinate course scheduling may impact students’ ability to complete the courses they 
need when they need them. Most colleges reported some way to make sure that classes 
would be offered to students during the semester they needed them or during a time of 
day that would be convenient to them. For example, faculty in one of the computer 
network technology programs scheduled the timing of the courses based on the 
demographics of the student body in the program. Because the program was targeted at 
students who were not working full-time, it was offered earlier in the day. College staff in 
other programs reported giving attention to the semesters in which a course was offered 
in order to make sure it was available when needed. This was sometimes accomplished 
through informal faculty coordination, particularly in small programs. In other cases, 
coordination was a result of the block scheduling of programs, where all classes during a 
particular time period were scheduled only for the program; other courses were typically 
offered at other times of the day. 

Most college staff reported both benefits and drawbacks to program prescription. 
Highly prescribed programs made it easier for students to navigate requirements and stay 
on track and easier for staff to manage the programs; they also created more consistency 
in students’ knowledge base, leading to better job preparation. However, highly 
prescribed programs were less flexible and less able to accommodate students’ 
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scheduling needs or to help a student recover from a failed course. Compared with the 
programs examined in other fields, the accounting programs had relatively low levels of 
prescription with no cohorts and no sequencing requirements. Accounting program staff 
reported that students were given the flexibility they needed but also given guidance on 
which courses to take. 

Overall assessment. The website review and the case studies of individual 
programs provide evidence that career-technical programs in the fields examined were 
highly prescribed. Most credits required to complete a program were mandatory (rather 
than elective) and program-specific (rather than general education) courses. Additionally, 
most programs (with the exception of accounting) gave students little to no flexibility in 
constructing their own programs. In fact, most of the programs we profiled used a 
cohort model. 

3.3 Information Quality 

A well-structured program should provide both prospective and current students 
with the information needed to make good decisions, and college websites are 
increasingly the primary information source for students. Thus, we examined the 
availability and clarity of information on program offerings to help students select 
programs of study and information on program requirements to help students complete 
programs. We used two data sources to better understand information quality: the website 
review and the case studies of individual programs. 

Unfortunately, our methods do not provide a perfectly effective way to evaluate 
information quality at each college. Because we interviewed administrators, faculty 
members, and counselors but not students, the interviews did not enable us to identity 
deficiencies in student knowledge that escaped the awareness of those with power at the 
college. Because lack of awareness of student knowledge deficiencies among colleges is 
a primary criticism made by Rosenbaum et al. (2006), this may be a particular concern. 
However, by examining program websites with an eye for what a prospective student 
would see, we can partially address this concern. 

Website review. During our website review, we looked at the availability and 
quality of information on key program characteristics. Our intention was to better 
understand — from the eyes of a prospective or current student — whether the website 
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provided the information necessary to decide whether the program was an appropriate fit 
and to understand the steps necessary to complete a credential. Appendix B shows the 
information we coded for each program website. The items we coded relate to key pieces 
of information that might be relevant to students as they decide whether to pursue a 
program and to program students as they make decisions about which courses to take, 
among other things. We assessed whether this information was available on the websites 
and how clearly the information was presented. 

Among the information that might be available, we examined whether the 
program website clearly listed program requirements — that is, which courses are required 
and whether there is a mandatory sequence in which they must be taken. We examined 
whether there was, at minimum, some type of table, graphic, or planning worksheet 
showing the courses that are required in order to graduate from the program. The college 
websites with the highest quality infonnation offered planning worksheets with required 
courses and elective course options, as well as a sample schedule that a student could 
take. Other websites did not include infonnation about which courses were required (for 
example, a website may have included a sample schedule without indicating which 
courses were required and which were electives) or had conflicting infonnation between 
the program website and the college’s official course catalogue. The presence of such 
discrepancies highlights the importance of ensuring that course requirements on the 
website are cunent and accurate. 

College websites should also ideally answer questions students may have about 
course scheduling: Ideally, how long does the program take to complete? Can students 
attend either full-time or part-time? How long will the program take to complete if a 
student has to take developmental coursework based on their placement test scores? In 
which quarters or semesters can the student begin the program? What time of day and 
which days of the week are courses offered? The websites with the highest quality 
information had the answers to these questions clearly listed and easily accessible. 

College websites should also include clear information about program pathways, 
such as how different credentials within the field of study are related; whether there are 
points where students can leave, gain an entry-level job, and then return to the program; 
and whether there are certificates that can be earned on the way to earning an associate 
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degree or a longer term certificate. Several excellent websites offered diagrams showing 
how different certificates and degrees were related, as well as the type of occupation (and 
mean hourly wages) that could be attained at each level; see Figure 3 for a generic 
example. Some colleges offered a number of credentials within a given field but did not 
provide much information to help a prospective student decide which would be most 
appropriate to pursue or whether the credentials were stackable (i.e., designed to 
articulate directly with higher level programs). 

The availability of information about prerequisites, admission requirements, and 
application procedures was difficult to evaluate because we had to assume that if no 
admission requirements were listed, the program was open to everyone. No websites that 
we reviewed provided current information about whether the program had a waiting list 
and its current length. If the program had a competitive admissions process, websites 
with high quality information clearly stated factors that affected admission decisions and 
provided a link to the program application form. If a program has prerequisites, these 
should also be stated clearly. For example, one career-technical program site included a 
printable checklist of courses that must be completed prior to program entry, along with 
grade requirements for each course. 


Figure 3 

Diagram of Certificate and Degree Progression with Related Work Opportunities 
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Program websites should offer information about students’ options for 
employment or continued education. Program websites should also offer information for 
prospective students about the kinds of occupations that program graduates work in, 
typical entry-level wages, and whether there is high labor market demand for students 
from the program. Websites with high quality infonnation listed jobs that recent 
graduates had obtained and provided data on how many students found work in their field 
and average wages for the occupation; other websites had no information on the topic of 
employment. If transfer to a baccalaureate institution is a possibility, program websites 
should offer infonnation about how many students transfer, whether there are 
requirements that students need to meet to be eligible for transfer, and whether there are 
articulation agreements with any nearby colleges for students to continue their education 
and earn a bachelor’s degree in a related field. The best websites li nk ed to separate web 
pages on the topic of transfer, explaining the additional requirements students needed to 
meet and listing the colleges to which students commonly transferred. Unfortunately, 
many colleges’ websites did not provide any information about transfer. It may be that 
these programs are not designed to lead to transfer, but ideally this would be made clear 
to prospective students. 

No matter how strong the website content, prospective students may have 
questions about the program. Most program websites provided adequate contact 
information, including the name, position, telephone number, and email address of 
someone with whom students could discuss the program. However, some websites had no 
contact information, information that may not have been current, or too many names 
listed without details to detennine who would be most appropriate to contact. 

To provide an overall assessment of the information quality of the colleges’ 
websites, we developed a numeric ranking system and ranked colleges on a scale of one 
to five. Program websites tended to have at least one significant problem that kept them 
from earning our top overall rating of five. Most websites had fairly high-quality 
information, though a couple lacked almost all useful information and earned our lowest 
rating of one. The average rating was approximately 3.2. 

We also examined websites for their accessibility in terms of the ease of finding 
information. Most college websites had a search function that worked effectively to 
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find program pages and requirements. We found that search functions worked 
appropriately more than 85 percent of the time. Because many students rely on the 
search engine tool to find information, programs should ensure that it works correctly 
for their web page. Site visitors should also be able to navigate to the program page and 
access infonnation on program requirements relatively quickly without a search engine. 
The fewer the number of clicks necessary to do so, the better. On average, about three 
clicks were necessary. 

Case studies of individual programs. The programs relied heavily on the college 
websites for disseminating information to students. Most, if not all, information necessary 
to choose programs and learn about program requirements was available online, 
including college catalogs, career guides, schedules of courses, and program planning 
guides. At several colleges, faculty, staff, and administrators used online tools to stay 
informed of program requirements and to track student progression. Most program 
websites generally had high-quality information but were missing some key infonnation. 

A strong website is important; websites were reported to be the primary source of 
information for students and, in many cases, college personnel. Interviewees reported that 
colleges were putting more resources into developing up-to-date websites and were 
directing students to the sites for information. Across the colleges, faculty and staff 
generally felt that students were getting the information they needed, although a few 
specific concerns were raised. However, as noted earlier, faculty and staff may not be in 
the best position to judge whether students are actually getting the infonnation they need. 

We observed a few differences by program area in the dimension of infonnation 
quality (see Table 6). In the automotive programs, faculty met with most prospective 
students before they entered the program and conducted a form of screening. In both 
accounting programs, faculty and staff relied heavily on program planning sheets for 
communicating infonnation to students. 

Information on program offerings. Students received infonnation on program 
offerings through college websites, new student orientations, counseling/advising centers, 
and worker retraining centers. Websites and counseling centers were the two main 
sources of information. Word of mouth was also mentioned as a way that students find 
out about programs. At least three of the program websites also had informational videos. 
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Table 6 

Manifestations of Information Quality in Individual Programs 
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Some college personnel suggested that because students who come to the colleges 
for career-technical work already have a strong idea of what they want to do, they may 
not need up-front infonnation about program offerings and career options. For example, 
in one of the medical assisting programs, many students knew they wanted to enter an 
allied health program and then chose a specific program after taking a few courses. In the 
computer networking technology programs, advisors and counselors found that almost all 
students who enrolled in the program knew what they wanted to study upon entry, though 
program coordinators reported that very rarely, students might decide among different 
offerings in a similar field (such as computer programming). 

Information on program requirements. In all of the programs, the college 
website was a major source of infonnation on program requirements. Several colleges 
aimed to develop their websites as the primary source for all infonnation. Most of the 
materials produced by the colleges were available online and accessed by students as well 
as faculty advisors and general advisors. In at least one program, advising was frequently 
done online through email. Most of the programs also made information available in the 
form of brochures, program handbooks and planning guides, and course catalogs. 

Program planning documents varied in their amount of prescription. For example, 
one accounting program planner provided a checklist of core and general education 
courses required to complete the credential but did not specify when during the two years 
the courses should be taken. The other program planner specified course requirements by 
tenn, although both accounting programs allowed for flexibility in course scheduling. 

In both automotive programs, prospective students were encouraged to meet 
with program faculty prior to enrolling to discuss requirements, tour the workspace, 
and determine if the program would be a good fit. Faculty at one of the computer 
network technology programs encouraged prospective students to visit the labs and 
meet with faculty. 

Communicating changes to program requirements. Methods for communicating 
program changes to advisors and others varied somewhat across the programs in our 
sample. Staff at four of the programs actively communicated changes by meeting with 
advisors or emailing them. At three programs, changes were communicated more 
passively through curriculum committees or program planning sheets. 
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Assessment of information provided to students, including website. Overall, 
respondents across the programs indicated that the appropriate information was available 
and that students were receiving the information necessary to enroll in and complete 
programs. Particularly in the cohort-based programs, respondents indicated that there was 
not much confusion among students because requirements were explicitly laid out and 
students were closely monitored by program faculty. This may indicate that information 
quality is a more important dimension of structure for programs with less program 
prescription — that is, programs that are not cohort-based and where faculty are not 
closely monitoring student progression. 

Although the overall assessment of information provided was positive, 
individual respondents from particular programs expressed some concerns. For 
example, in one accounting program, an instructor indicated that students were 
sometimes confused about the differences between the terminal accounting degree and 
the transfer business degree. Financial aid information was a concern across several 
programs. An advisor for one college indicated that students had trouble accessing 
information and struggled in particular with financial aid information. At another 
college, a dean stated that students did not receive timely information to complete 
financial aid forms before deadlines. 

Overall assessment. We found mixed evidence regarding information quality. In 
particular, program websites, although decent overall, sometimes lacked certain 
information that might be important for prospective or current students. This is especially 
concerning because program staff indicated that websites were the primary source of 
information for students. However, in interviews, program administrators, counselors, 
and faculty members generally were positive about the state of infonnation quality and 
did not see need for improvement. Because we did not interview students, it is difficult to 
determine precisely how effective current efforts at presenting information students are. 
For example, students in programs with cohort models may not have a strong need for 
information because it is provided in the context of their courses; however, this 
information may still be important for prospective students. 
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3.4 Active Advising and Student Support 

Active advising and student support refers to the actions that college staff take to 
assist students with enrolling in, progressing through, and completing programs. As with 
the other dimensions of structure, we hypothesized that colleges in which this dimension 
was strong would have students who were more likely to persist and attain a credential. 

To better understand this dimension of structure, we relied on data from our case studies 
of individual programs. 

Case studies of individual programs. The eight colleges we studied had highly 
prescribed programs, reducing the amount of individual guidance and support students 
needed for academic planning. For the most part, students were closely monitored and 
supported by program faculty, with whom they often spent extensive amounts of time. 
This was especially true for students in programs with a cohort structure. 

College advisors and counselors also provided a considerable amount of support, 
especially before students decided which program to enter. Program l’s college had 
program-specific advisors. Some students were supported by special programs such as 
WorkFirst, and some colleges employed early warning systems for all their students. In 
general, there was more variation in the guidance and support received by students not 
yet enrolled in programs. Table 7 provides details on different aspects of this active 
advising and student support in the eight colleges. 

Counseling/advising model. In all of the colleges studied, once students 
entered a program, they were advised by the program faculty. In smaller programs, this 
was sometimes done by the one or two full-time faculty in the program. In the 
computer network technology programs, students were advised by whichever faculty 
member taught their first-quarter class. In most programs, especially those with a 
cohort model, faculty members spent 30 hours or more per week in the classroom with 
their students. These faculty members often developed strong relationships with 
students and supported them in completing the requirements of the program. Students 
participating in special programs such as WorkFirst received additional support from 
staff in these programs. 
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Information for undecided students provided by programs. For the most part, 
undecided students were assisted by general counseling staff at the college who talked 
with them about their options. Student orientations also helped some students to clarify 
their goals and program choice. Program staff were more likely to become involved when 
students narrowed their choices to a general field of study. In the case of allied health, the 
program or division offered classes or orientations to help students select among different 
health careers. In computer network technology and automotive technology, prospective 
students met with faculty members and visited the labs. This provided students with an 
opportunity to discuss their interests, goals, and aptitudes and whether the program would 
be a good fit. The two medical assisting programs offered extra opportunities for 
undecided students who needed more information about the range of allied health 
programs — in one case, a class was offered; in the other, an orientation. 

Group sessions for advising. Group advising sessions were the nonn across all 
colleges and fields of study. These sessions were generally designed to ensure that all 
students in a program registered for the appropriate courses in the subsequent term to stay 
on track toward completing their programs. In addition, individual advising sessions with 
faculty were often available when needed. 

Program orientation. Most of the colleges offered a general orientation session, 
but there was variation in the use of program-specific orientations. Medical assisting and 
automotive technology programs had a separate orientation, but computer network 
technology and accounting programs did not. Notably, one program had a substantial 
group of non-native English speaking students who entered via a summer program 
(which had its own orientation), whereas another program had a one-credit summer 
orientation program for entering students. 

Monitoring of student progress. At most colleges, faculty members were very 
involved with program students and monitored their progress closely. Monitoring was 
more proactive in computer network technology and automotive technology programs 
and less so in accounting and allied health programs. In addition, several colleges had or 
were developing early warning systems to allow for identification of and early 
intervention for struggling students. 
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Support for struggling students . At the institutional level, all colleges offered 
tutoring in math and English for all students. This was a priority for colleges because, 
like students across the country, students in the programs we studied often struggled with 
math in particular. Colleges also offered a range of other supports, which differed by 
college, program, and the presence of support programs such as TRIO or WorkFirst. 
Students associated with special programs often received case management services, 
extra tutoring, and specialized counseling. Several programs also made arrangements for 
more advanced students to assist struggling students. 

Overall assessment. To some extent, the quality of advising and student supports 
was related to whether or not programs used a cohort model. With the cohort model (used 
by five of the eight programs we profiled) supports are built into the model of instruction. 
Professors spend time with students during substantial class sessions sometimes every 
weekday, get to know students closely, and are able to easily monitor student progress 
throughout the entire program. Additionally, in cohort model programs, there is typically 
only one course for students to register for the next semester, and students are easily 
advised along those lines. In programs without a cohort model, we found a wider range of 
advising and student supports. Colleges took additional measures to ensure student 
success, including implementing early warning systems, having program-specific 
advisors, and offering group advising days. Using strategies such as these, programs can 
build advising and support into the structure of the program so that all students receive 
additional support, even if they do not actively seek it. 


4. Implications of Structure for Program Performance 

4.1 Limitations of Our Performance Measures 

Although our research design involved selecting colleges for qualitative research 
by assessing program performance, we faced challenges in comparing performance 
across specific programs. This section presents an analysis of differences in perfonnance 
across fields of study based on our case studies, with several caveats. First, the 
performance data are based on data from the 2005-06 student cohort, so they reflect 
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program operations from up to five years prior to our case studies. Although in our 
interviews we asked respondents to describe any major changes that had taken place 
since then, we cannot guarantee an accurate description of how those programs operated 
in the past because of the time elapsed and in some cases personnel turnover. 

Second, because of the limited sample sizes at the program level, we calculated 
the program performance data on the rates of long-term credential completion at the level 
of the field of study. Because the field of study typically includes several programs, our 
ability to directly observe the link between structure and performance for particular 
programs is limited. However, within fields of study, we typically selected programs that 
had the most completers and therefore were likely to be relatively large programs that 
substantially drove the overall outcomes for a field of study. On the other hand, this could 
also bias our selection toward more successful programs within a field of study. 
Therefore, we cannot attribute field-level completion rates to program-level performance, 
though the field-level completion rates are certainly suggestive. 

Third, our program performance measures are based on the idea that receiving 
any long-tenn certificate or associate degree constitutes success. Programs that operate 
under a different definition of success — such as receiving any certificate, even a short- 
term one, or only receiving an associate degree — could be at a disadvantage in these 
rankings. Prior research suggests that the proportions of different types of credentials 
awarded vary substantially across colleges (Scott-Clayton & Weiss, 201 1). 

Although we recognize that these caveats limit the certainty with which we can 
link our case studies of structure in our programs with field of study performance data, it 
is reasonable to expect there is a degree of connection that allows us to explore possible 
differences in performance in relationship to program operations. These limitations are 
inherent to this type of research focused on specific programs — the type of focus that is 
necessary to understand how programs are structured. 

4.2 Findings 

In general, we did not observe differences in the structure-related practices of 
high- and low- performing programs that would account for their differences in 
performance. In our interviews with faculty, administrators, and counselors, we found no 
substantial differences in the extent of program structure across colleges within fields of 
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study. We did observe different models of program structure — that is, different ways that 
programs offered a structured experience for students — but all of the career-technical 
programs we examined were either highly or moderately structured. The only variation 
we found in the extent of program structure was between fields, with accounting 
programs being somewhat less structured than the others. 

Data from our website review supports the finding that program perfonnance is 
uncorrelated with the degree of program structure, particularly with respect to information 
quality. Table 8 displays a summary of website accessibility information that was collected 
for the high- and low-perfonning colleges. We found no significant differences between the 
two types of colleges on these scores; raw scores for the low-perfonning colleges were 
slightly but not significantly higher on these measures of infonnation quality. 


Table 8 

Accessibility of Information at High- and Low-Performing Colleges 


Measure 

High-Performing 

Low-Performing 

Colleges 

Colleges 

Number of clicks to get to program requirements 

3.10 

2.97 

Overall rating 

3.08 

3.30 

Search function utility 

0.83 

0.91 


Source: College websites. 


Similarly, we did not find consistent patterns across high- and low-performing 
programs in the number of required versus elective credits or the number of program 
versus general education credits. However, suggestive differences emerged between 
programs at colleges that were assessed to be high- and low-perfonning within three of 
our four fields of study (in the accounting programs, no major factors that are part of the 
structure hypothesis explain the observed differences in performance). 

Across the two automotive programs, differences in college-level approaches to 
working with students may explain some of the difference in performance. The higher 
performing program was in a prize-winning college, noted for its practices to promote 
student success that make it outstanding on the national level. 

In the two medical assisting programs, differences in college-level approaches to 
working with students may explain some of the difference in performance. The high- 
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performing college had proactive systems in place to support students, including an early 
alert system to identify students who were having problems and provide them with extra 
support as soon as possible. It also offered a course on allied health careers to help 
students make better decisions about which program to enter, which could lead to better 
completion rates if students initially enter programs for which they are better suited. In 
addition, the two programs used different counseling models. The high-performing 
program relied primarily on a dedicated counselor and an advisor in the allied health 
fields rather than faculty advising. Both of these staff members were located in the same 
area as others in the allied health field and worked exclusively with those students and 
faculty; they were integrated into the allied health field. In the other college, the 
counseling was more general and less focused on specific fields of study. 

In computer network technology, the high-performing program’s emphasis on 
immediate entry-level job placement versus associate degree completion might partially 
explain why its outcomes exceeded those of the lower performing program. Prior 
research suggests that links to labor market opportunities motivate students to engage in 
and complete a program of study (Rosenbaum et al., 2006). 

4.3 Summary 

Across the fields of study, a few emerging themes help to explain the differences 
in perfonnance across the eight programs. First, we found suggestive evidence that a 
college-level emphasis on providing proactive support to students might be beneficial. 
Programs in colleges with additional systems to engage and support students were more 
likely to be high-performing. A common college-level mechanism in high-performing 
colleges was an early alert system, which provides a proactive and potentially consistent 
way to identify students who are having trouble with a course or a program of study and 
intervene before they fall too far behind. The existence of this system in a college likely 
reflects the a high level of institutional commitment to engaging with students and 
providing support to promote credential completion. 

Second, we found a promising practice at one high-performing college: 
employing dedicated counselors specific to the field of study instead of relying on faculty 
advisors or general counselors. The program had dedicated allied health counselors who 
were deeply knowledgeable about the program and dedicated to advising students in the 


36 



specific program. Because of the small sample size and because we only found one 
program with this practice, we cannot make generalizations about whether this practice 
leads to better outcomes. However, the relationship between counseling approaches and 
student completion is a promising area for further investigation. 

Last, we noticed that the higher performing programs were less likely to 
emphasize the associate degree (as compared with long-term certificates that require 
fewer general education courses) and tended to offer fewer short-term certificates. These 
differences could be causally related to differences in completion rates. For example, an 
emphasis on earning a long-tenn certificate and then immediately seeking paid 
employment could provide students with more motivation to complete than a bigger- 
picture focus on an associate degree to improve long-tenn career options. Alternatively, 
these differences could be an artifact of our ranking mechanism. Students who only 
intend to get a short-tenn certificate are not considered successes in our ranking model, 
and long-term certificates and associate degrees are weighted equally even though 
associate degrees typically have more requirements. It is also possible that programs only 
began offering short-term certificates because many students were not graduating, so that 
students would have at least some credential even if they dropped out. 


5. Conclusion and Recommendations 

All of the programs we examined exhibited fairly high levels of structure across 
the four dimensions in our framework: program alignment, program prescription, 
information quality, and active program advising and support. In practice, however, these 
dimensions of structure were manifested in different ways by different programs. The 
eight career-technical programs we profiled were from the most popular fields of study 
among the community and technical colleges across the state of Washington, so our 
results provide infonnation that is applicable to a large number of students pursuing 
career-technical programs. 

None of the programs we studied were loosely structured; all were either 
moderately or highly structured. Table 9 summarizes these findings across the four 
dimensions of structure at each of the eight programs. Six of the eight programs we 
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studied in depth were highly structured, all of which were in fields of study (allied health, 
computer and information science, and mechanics and repair) where occupational or 
industry licensing or skill standards strongly influenced the programs. As a result, 
program requirements were highly prescribed, labor market linkages were fairly tight, 
and there were strong student supports either embedded into the program or otherwise 
structured into the student experience. 

The two accounting programs were moderately structured. They had a moderate 
amount of program prescription but offered some flexibility in scheduling and elective 
courses; their labor market linkages were weaker than those observed in other career- 
technical programs; and they did not have particularly proactive counseling and advising 
practices. To some extent, these programs’ practices may have been guided by the 
structures and norms associated with the field of accounting, which lacks strict industry 
standards or guidelines, at least at the sub-baccalaureate level. The lack of specific 
industry and occupational mandates for programs may be reflected in greater flexibility in 
how the programs are organized, such as the absence of a cohort model. In these ways, 
the accounting programs were more similar to other programs at the college that were 
focused on preparation for baccalaureate transfer and likely to have less structure and 
fewer labor market linkages. 


Table 9 

Levels of Program Structure at Each Program by Dimension 



Medical Assisting 

Accounting 

Computer Network 
Technology 

Automotive 

Dimension 

Program 1 

Program 2 

Program 3 

Program 4 

Program 5 

Program 6 

Program 7 

Program 8 

Overall level 
of structure 

High 

High 

Moderate 

Moderate 

High 

High 

High 

High 

Program 

alignment 

High 

High 

Low- 

moderate 

Low- 

moderate 

Moderate 

-high 

Moderate 

High 

High 

Program 

prescription 

High 

Moderate 

-high 

Moderate 

Moderate 

High 

High 

High 

High 

Advising 
and student 
support 

High 

Moderate 

-high 

Moderate 

Moderate 

High 

High 

High 

High 

Information 

quality 

Moderate 

-high 

Moderate 

-high 

Low- 

moderate 

Moderate 

Moderate 

Low- 

moderate 

Moderate 

Moderate 

-high 
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Program prescription was the dimension for which we found the least variation in 
structure. Five of the eight programs we examined followed a cohort model, in which 
both the courses required and the sequence in which they should be taken were mandated, 
with cohorts progressing through the sequence together. (In some cases, there were still 
opportunities for students to pursue electives after the daily schedule or over the 
summer.) In the medical assisting program that did not follow a cohort model, there was 
still relatively little flexibility in the program, and course prerequisites caused natural 
sequencing to occur. In the accounting programs, most courses were mandatory, but there 
were more opportunities for students to customize their programs. 

Programs were also strongly aligned with labor markets and local employers. 
Labor market alignment was generally achieved through accreditation agencies, advisory 
boards, or both (with the partial exception of the accounting programs). Program staff 
also structured job search and job placement activities directly into the programs, at least 
to a moderate extent. Programs varied, however, in the degree to which they were aligned 
with further educational opportunities, particularly baccalaureate transfer; in general, 
most programs were viewed as terminal degrees. 

In some ways, the highly prescribed nature of the programs obviated the need for 
highly structured infonnation delivery, advising, and student support. Students within 
highly prescribed programs are given less information and guidance because options tend 
not to be overwhelming or confusing. That said, some programs offered strong supports, 
structuring advising and counseling directly into the student experience through early 
alert systems, program-specific advisors, and group advising days. 

Overall, community and technical college career-technical programs in the fields 
we examined were highly structured in terms of program alignment, program 
prescription, and student supports (though we found room for improvement in the 
dimension of information quality). In contrast with the community college programs 
examined in Rosenbaum et al. (2006), our sample consisted primarily of highly 
prescribed programs with strong connections to labor markets and local employers 
through accreditation bodies and advisory boards. It therefore may not be possible, in 
general, to significantly increase the level of prescription among most career-technical 
programs (assuming that they are organized like the ones examined here). 
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The majority of career- technical programs we examined in our case studies 
followed a cohort model, which naturally imposes structure on students along several 
dimensions. Students in programs with a cohort model are given a single path toward a 
credential that does not leave room for mistakes and given clear information and support 
from instructors who spend considerable class time with them each week. On the other 
hand, by not leaving room for students to make mistakes, cohort models could adversely 
affect completion rates by imposing too many requirements for some students to 
realistically meet or by making it difficult for students who make mistakes to continue in 
the program. 

We found that career-technical programs at Washington community and technical 
colleges were not unstructured or overly bureaucratic, as community college programs 
are sometimes characterized to be (see, e.g., Rosenbaum et ah, 2006). Some previous 
research has found suggestive evidence that traditional liberal arts, transfer-oriented 
programs might be substantively different from (and less structured than) career-technical 
programs at community and technical colleges. For example, Zeidenberg and Scott 
(2011) analyzed the transcripts of a cohort of first-time students in Washington 
community and technical colleges and found that the courses taken by career-technical 
students clustered easily into discrete career-technical programs, but liberal arts clusters 
were not clearly distinguishable. This suggests that liberal arts students may not have 
been following prescribed pathways. It may be that even within community colleges, 
liberal arts programs are more loosely structured than career-technical programs, which 
are highly prescribed and tightly aligned with local labor markets through accreditation 
bodies, advisory boards, and industry certifications. 

The career-technical programs we examined tended to score more poorly on the 
dimension of information quality, particularly with regard to prospective students who 
may not have been receiving information directly from program faculty. Without student 
interview or survey data, it is difficult to assess how well infonnation was reaching 
students. However, this may be less of a concern for career-technical programs than at the 
college more generally; advisors reported that students entering career-technical 
programs typically entered the college knowing which program they wished to enroll in. 
That said, colleges should ensure that appropriate information is available to students, 
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particularly through the college website. In the Section 3.3, we provide specific 
suggestions for elements that should be included in all program websites. Programs 
should also ensure that the information that is available is correct and up-to-date. 

Overall, we found limited evidence of a connection between program structure 
and program perfonnance. One reason for this is that both the high- and low-performing 
programs we examined tended to be relatively highly structured across the four 
dimensions of our framework. Structured programs might be a necessary condition for 
success, but given the relative uniformity of structure we observed in these career- 
technical programs, the observed differences in their success may be due to other factors. 
In addition, our performance metrics, which were measured at the program area level, 
may have limited ability to accurately gauge the relative performance of specific 
programs within those broader program areas. It could be that the measures we used did 
not effectively distinguish between higher and lower performing programs, or that the 
measures reflected other factors, such as program length. To the extent that the measures 
accurately measure program performance, the evidence does not support the hypothesis 
that more highly structured programs explain differences in student outcomes. We did 
observe some differences between high- and low-performing programs through our 
fieldwork, but these were related to college-wide student support practices rather than 
practices specific to the programs themselves. Given that the main focus of this study was 
on the organization of programs rather than college supports more generally, our 
suggestions about the possible reasons for the differences between high- and low- 
performing programs are at best conjecture. 

For the reasons described above, structure may not be a particularly productive 
element for career-technical programs to focus on as they attempt to improve completion 
rates because the programs we observed, at least, were already highly structured. It may 
be more important to help more students enter highly structured programs and to improve 
structure in transfer programs in liberal arts or business programs. 
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Appendix A: Site Selection Methodology 


Because issues of structure are most relevant to programs that take some time to 
complete, this study focused on the completion of long-term credentials. We examined 
the proportion of students from the 2005-06 cohort who attempted a concentration within 
a given field of study at the college who earned either a long-tenn certificate or an 
associate degree by the end of the 2008-09 school year. We consider long-tenn 
certificates to be those that require one year or more of full-time study (45 quarter 
credits). We exclude short-term certificates (which can be completed with less than one 
year of full-time study) because research suggests they may have relatively little value in 
the labor market (Dadgar & Weiss, 2012; Jepsen, Troske, & Coomes, 2012). As a result, 
programs that specialize in short-term certificates may appear to have lower completion 
rates based on our calculations. 

Using completion rates of long-term credentials as the outcome, we perfonned a 
multivariate regression using student-level data to predict expected college completion 
rates. All students who attempted a concentration in a given field of study were included 
in the regression, even if they were enrolled at a college that did not have a large enough 
sample size to be considered for selection. We then calculated the difference between 
each school’s actual and predicted completion rates and ranked colleges by this 
difference. An institution performed highly if its actual completion rate exceeded the 
completion rate that would be expected based on its student characteristics (and selected 
school characteristics). 

Table A. 1 summarizes the results of this analysis for colleges with at least 20 
students in a given field. The first three columns summarize colleges’ actual completion 
rates, where colleges varied widely in terms of success. The next three columns 
summarize colleges’ predicted completion rates, or the completion rates that would be 
predicted for each college using the results from our multivariate regression, controlling 
for student characteristics. The final three columns summarize the magnitude of this 


s Student characteristic controls included age, race, enrollment status (part-time or full-time) in the first 
quarter, Pell grant receipt, self-reported intent, and socioeconomic status based on census block 
characteristics. School characteristic controls included the degree of urbanization of the college’s location 
and the percentage of minority students. 
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difference — that is, how much colleges overperformed or underperfonned in relation to 
their expected completion rates. For example, in the allied health field, the college that 
performed the best relative to its projected outcome had a predicted completion rate of 
31.4 percent and an actual completion rate of 52.7 percent — a difference of 21 percentage 
points. This difference is the value that was used to determine college performance for 
the purpose of site selection. Because completion rates were calculated separately for 
concentrators in each field of study, a given college might perform well in one field but 
poorly in another. 

For our website reviews, we selected 20 colleges across the four fields of study, 
including both relatively high- and low-performing colleges. For our interviews, we 
selected two colleges in each of the four fields of study, including one relatively high- 
performing college and one relatively low-performing college. We selected the cases 
that maximized variation in program performance to obtain the most information-rich 
sources of data for our preliminary examination of the link between structure and 
credential completion. 


Table A.l 

Actual and Predicted Long-Term Credential Completion Rates Across Colleges, 
Based on Value-Added Measures 


Field of Study 

Actual Completion 
Rate (%) 

Predicted Completion 
Rate (%) 

Difference (%) 

Min 

Max 

Mean 

Min 

Max 

Mean 

Min 

Max 

Mean 

Allied health 

5 

55 

32 

20 

47 

31 

-19 

21 

1 

Business and marketing 

0 

35 

19 

1 

30 

20 

-17 

10 

0 

Computer and 

0 

45 

13 

2 

25 

13 

-11 

21 

0 

information sciences 










Mechanics and repair 

6 

73 

30 

21 

34 

27 

-18 

40 

3 


Source: SBCTC administrative data 
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Appendix B: Coding Guidelines for Program Websites 

Table B.l 

Information Quality of Websites: Items Coded 


Item Coding Guidelines 

Table or graphic on program requirements Indicate if this is available and, if so, provide a general 

description 

Sequence for course taking Indicate if this is available and, if so, provide a general 

description 

Prerequisite information Information on prerequisites is listed (yes/no) 


Transfer information listed 

Employment information 

Program performance information 
Contact information for students 


Including information on opportunities and requirements for 
transfer 

Including information on job prospects, placement 
assistance, anything employment related 

Including number of credentials awarded, completion rate 

Where to call for more information on program 


Certification or licensure associated with program Indicate if an industry certification or state licensure is 

associated with the program, list the name 

Application required Indicate if the program is selective, requires admission 


Source: College websites. 
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